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Abstract 

Eight pilots participated as test subjects in a fixed- 
based simulation experiment to evaluate advanced 
vision display technologies such as Enhanced Vision 
(EV) and Synthetic Vision (SV) for providing terrain 
imagery on flight displays in a Lunar Lander Vehicle. 
Subjects were asked to fly 20 approaches to the Apollo 
15 lunar landing site with four different display 
concepts - Baseline (symbology only with no terrain 
imagery), EV only (terrain imagery from Forward 
Looking InfiraRed, or FLIR, and Light Detection and 
Ranging, or LIDAR, sensors), SV only (terrain 
imagery from onboard database), and Fused EV and 
SV concepts. As expected, manual landing 
performance was excellent (within a meter of landing 
site center) and not affected by the inclusion of EV or 
SV terrain imagery on the Lunar Lander flight 
displays. Subjective ratings revealed significant 
situation awareness improvements with the concepts 
employing EV and/or SV terrain imagery compared to 
the Baseline condition that had no terrain imagery. In 
addition, display concepts employing EV imagery 
(compared to the SV and Baseline concepts which had 
none) were significantly better for pilot detection of 
intentional but unannounced navigation failures since 
this imagery provided an intuitive and obvious visual 
methodology to monitor the validity of the navigation 
solution. 

Introduction 

Past aviation research has demonstrated the 
substantial potential of synthetic vision (SV) and 
enhanced vision (EV) technologies to enable 
equivalent visual operations independent of actual 
weather and visibility conditions [1-4]. SV is a 
computer-generated image of the external scene 
topography, generated from vehicle attitude, high- 


precision navigation, and data of the terrain, obstacles, 
cultural features, and other required flight information 
presented to a pilot on his/her flight displays (e.g., 
head-up and head-down primary flight displays, 
navigation displays, and auxiliary displays such as 
electronic flight bags). EV is an electronic means of 
providing a display of the external scene by use of an 
imaging sensor, such as Forward-Looking InfiraRed 
(FLIR) or Light Detection and Ranging (LIDAR). By 
creating, supplementing, or enhancing the natural 
vision of the pilot, these vision-based technologies 
have been shown to significantly improve situation 
awareness, reduce workload, enhance aviation safety 
(e.g., controlled-flight-into-terrain prevention and other 
hazards detection), and promote flight path control 
precision. 

The intended use of EV and SV technologies 
mirror each other, as they both attempt to eliminate 
low-visibility conditions and replicate the operational 
benefits of clear-day flight operations, regardless of the 
actual outside visibility conditions. The issues that 
drove the design and development of synthetic and 
enhanced vision have commonalities to other 
application domains; most notably, during descent and 
landing on moon and planetary surfaces. 

To investigate the potential benefits of an EV 
and/or SV-enhanced flight deck for spacecraft, the 
Apollo lunar landing mission was reviewed. The 
Apollo astronauts were tasked with navigating a new 
vehicle, the Lunar Module (LM), in an unknown 
environment with basic flight instrumentation, mainly 
consisting of electro-mechanical gauges (Figure 1). 
Astronaut Neil Armstrong identified the landing phase 
as being the most difficult part of the Apollo 11 
mission [5]; therefore, this experiment chose to focus 
on the descent to landing stage. Also, Apollo 
experience showed that [6] “the constraints placed on 



crew visibility by the design of the LM window and by 
trajectory parameters make the viewing of the 
programmed landing site a major problem.” For the 
Apollo missions, pilot visibility was of the utmost 
importance. The lunar landing task relied on the 
pilot’s natural vision, coupled with large forward- 
facing windows and operational flight profiles which 
allowed the pilot to see the landing zone for extended 
periods of time, to perform a visual, manual landing 
approach or to re-designate and fly to a new landing 
area if the original landing site was not suitable. The 
Apollo LM windows provided almost 70 degrees nose- 
down visibility. 



Figure 1. Apollo Lunar Module 


The importance of pilot visibility was not only 
emphasized by trajectory design and window 
definition, but also, by conducting the Apollo landing 
task only at specific times and locations to provide 
optimal sun light on the landing site [5]. The mission 
was designed around lighting conditions (typically 
lasting about a week) that would create shadows 
providing optimal depth perception. 


In response to a 2004 Presidential directive to 
“implement a sustained and affordable human and 
robotic program to explore the solar system and 
beyond” [7], NASA initiated the Constellation 
Program. NASA’s Constellation Program is 
developing a new space transportation system to travel 
beyond low Earth orbit to initially establish a sustained 
human presence on the moon and then go to Mars and 
other destinations. Constellation is developing the 
Orion Spacecraft, the Altair Lunar Lander, and the 
Ares I and V rockets to take humans to the moon to 
establish a lunar outpost. 


Preliminary concepts and operations [8-9] 
anticipated for the Altair Lunar Lander are 


contradicting Apollo’s experience in the use of a 
pilot’s natural vision as the primary landing aid. First, 
preliminary design concepts for a Lunar Lander show 
significantly less external visibility for the astronauts 
than that provided to the Apollo astronauts. The Altair 
design will be significantly larger than the Apollo LM 
and the size of the windows and their location may be 
severely constrained and non-optimal for pilot 
visibility. Second, fuel-optimized trajectories do not 
include a “pitch-up” maneuver, as Apollo did, for pilot 
visibility through windows. If the need for natural 
visibility could be eliminated or reduced, significant 
fuel savings could be gained. Finally, future mission 
requirements call for frequent landings in the Lunar 
polar regions for maximum scientific benefit. The 
polar regions exhibit less than ideally lighted lunar 
surface locations. The very low sun-angles will create 
shadows that can mask landing hazards to a pilot’s 
natural vision, such as craters and boulders. Each of 
these factors emphasizes the importance of providing 
technological capabilities for unfettered moon surface 
landings, without sole reliance on a pilot’s natural 
vision. 

Based on research from the aviation domain, SV 
and EV technologies hold significant promise for 
planetary missions because the technology presents a 
computer-generated (SV) and real-time (EV) view of 
the terrain, topography and other significant 
characteristics (e.g., habitat modules, power 
generators, etc.) independent of the outside visibility 
conditions (i.e., darkness, shadow illusions or 
obscuration, engine effluent/dust), window locations, 
or vehicle attributes. SV allows unconstrained control 
of the computer-generated scene lighting, terrain 
coloring and virtual camera angles which may provide 
invaluable visual cues to pilots/astronauts, not 
available from other vision technologies. EV sensors, 
such as FLIR and LIDAR, provide real-time imagery 
of surface topography and hazards and can provide 
real-time integrity monitoring of the SV database. 
Vehicle state information (e.g., forward and down 
velocities, altitude, and fuel remaining) may be 
conformally overlaid on the SV/EV imagery to 
enhance trajectory control and overall crew situation 
awareness especially when compared to the 
information presentation and data display used for the 
Apollo LM. 




Current Study 

The objective of this study was to gain a first- 
order assessment of whether the benefits of EV/SV 
technologies, shown in the aviation domain, do indeed 
carry over to the space domain. This experiment 
examined the effects of having terrain imagery on 
space vehicle flight displays during lunar approaches 
and landings for nominal and off-nominal situations. 

The terrain imagery concepts were: (a) EV-only, 

(b) SV-only, (c) EV and SV, and (d) Baseline (i.e., no 
SV or EV) Head-Down Display Concepts. These 
terrain imagery concepts were shown on head-down 
displays providing a head-up display (HUD)-like 
format, primary flight reference information display, 
navigation display (ND), and an ego-centric situation 
awareness (SA) information display. 

The specific research objectives included: (a) 
evaluating the contribution of EV and/or SV 
visualization technologies on vehicle flight displays for 
approaches and landings to the lunar surface; (b) 
evaluating the effects of various vehicle external field- 
of-views and EV and/or SV visualization technologies; 

(c) evaluating SA and workload during terrain- 
challenged, fuel-optimized approaches; and (d) 
evaluating the contribution of terrain and hazard 
display depiction on landing redesignation 
functionality. 

Method 

Subjects 

Eight pilots participated as subjects in this 
experiment. Because of the powered-lift nature of the 
Lunar Lander vehicle and the vertical descent and 
landing task, rotorcraft piloting experience was felt to 
be important to reduce training time for the simulation. 
Pilot subject recruitment was conducted, using 
established NASA protocol, requiring subjects to have 
a commercial pilot’s license, and if possible, rotorcraft 
experience. Six of the evaluation pilots (EPs) had 
previous experience flying rotorcraft with an average 
of 687 hours of flight time. 

Simulation Facility 

The experiment was conducted in the Visual 
Imaging Simulator for Transport Aircraft Systems 
(VISTAS) III rapid-prototyping workstation at NASA 
Langley Research Center. VISTAS III is a non-motion 


platform, nominally used for crew-vehicle interface 
concept development for transport aircraft (Figure 2). 



Figure 2. VISTAS III Simulator 


For this single pilot experiment, a Logitech Freedom 
joystick was added for the pilot interface to the lunar 
lander simulation and had pitch/roll/yaw attitude 
control, thrust control and 10 programmable buttons 
[10]. The head-down display concept evaluations were 
shown on a 1600x1200 liquid crystal display flat panel 
with a 22 inch total display diagonal. Four equal-sized 
display areas (9 inches wide by 7 inches high) provided 
four flight instrument formats: HUD-like information, 
Primary Flight Display (PFD) information, ND 
information, and SA information. 

The out-the-window (OTW) view consisted of three 
1024x1280 displays with a total resolution of 3072 
pixels horizontal by 1280 pixels vertical. The total 
field-of-view was 72° horizontal by 30° vertical with a 
resolution of 43 pixels/degree. The out the window 
displays were centered vertically to the pilot design eye 
point giving an equal 15° up and down look angle. 
Note that this look-down angle of 15° is a limitation of 
the VISTAS III facility as it doesn’t allow the 70° 
look-down angle of the Apollo windows. 

Vehicle and Lunar Properties 

Available public-release data on Lunar Lander 
concepts were used to derive mass and inertial 
properties, from Lunar Orbit Insertion, to the 
appropriate trajectory initialization point for each run, 
including nominal Reaction Control System fuel 
utilization, to that point. The Moon was simulated as a 
homogeneous, spherical, rotating mass, for purposes of 
this experiment. A fourth order Runge-Kutta 


integrator was used for state propagation, with a fixed 
step size of 0.01 seconds. 

Lunar Database 

The Apollo 15 landing site was utilized in this 
experiment because of the availability of higher 
resolution data and the interesting terrain features it 
provided. The visual lunar database was created using 
lunar topology and imagery data collected by the 
Clementine project. Clementine was launched on 
January 25, 1994 and collected lunar surface data 
during the months of February, March, and April of 
that year. A spherical lunar database was created that 
covered the entire lunar surface and was built with 
three topology and imagery resolutions. Database 
coverage for the entire moon was created using a 
resolution of approximately 4 kilometers per elevation 
post, draped with 1 kilometer per pixel imagery. 
Medium resolution data was used around the area of 
the Apollo 15 landing site, with approximately 500 
meters per elevation post, draped with 125 meter per 
pixel imagery. The highest resolution data was used in 
the area immediately surrounding the Apollo 15 
landing site. This resolution was 15 meters per 
elevation post draped with 4 meter per pixel imagery. 

The same terrain data were used for the SV, FLIR, 
and OTW presentations; however, their display 
presentations were altered to roughly approximate the 
different sensors and concepts. For the FLIR 

presentation, the imagery color was converted to green 
based on the intensity of the true color and the 
brightness was attenuated based on pixel distance and 
elevation/azimuth angle. Random noise was also 
inserted. The SV and OTW presentations were 
identical except for the addition of boulder hazards in 
the OTW scene. Fifty boulders were pseudo-randomly 
placed around the Apollo 15 landing site in a 0.015 
degree by 0.015 degree area (Figure 3). The boulders 
ranged from 0.5 meters in diameter to 10 meters in 
diameter. The same boulders were also visible in the 
FLIR presentation. 

A surface feature (hereinafter referred to as the 
lunar landing cue) was placed at the landing site so the 
EP’s would have a visual reference to the desired 
touchdown point (see Figure 3). The lunar landing cue 
was a circular ring that was 60 meters in diameter, 10 
meters thick, and two meters high. The ring was 
textured with a pseudo lunar surface image and 
blended into the lunar surface topology. The lunar 


landing cue was displayed in all three terrain 
presentations (SV, EV FLIR, and OTW). 



Figure 3. Lunar Landing Cue in OTW View 


SV Database 

As mentioned in the lunar database section, the 
SV database presentation of lunar terrain was identical 
to that provided OTW except it did not include the 
boulder hazards. The boulders were not present in the 
SV terrain to simulate objects which might be too 
small to be detected by a terrain data gathering sensor. 
Further, as the SV terrain is computer generated, actual 
sun angles do not affect the visibility of the terrain. 

EV Sensors 

Two rudimentary, non-physics-based sensors 
were simulated in this experiment: FLIR and LIDAR. 
The FLIR presentation was identical to the OTW view 
of the lunar terrain, including the boulder hazards and 
the lunar landing cue. It was depicted in green on the 
display concepts using EV sensor information (see EV 
and Fused EV/SV display concepts in Figure 4). The 
LIDAR sensor provided a “flash shot” of the lunar 
terrain at a 1000 m altitude on approach. The LIDAR 
data was not shown directly to the EP, but instead, a 
simulated “processed” image of computed hazards was 
shown. “LIDAR-detected” hazards were depicted as 
transparent, red colored circles (Figure 4) on the 
display concepts that used EV sensor information. 
These LIDAR-detected hazards were only displayed 
after the vehicle had passed through 1000 m altitude. 


Independent Variable - Display Concepts 

Four flight display concepts (Figure 4) were 
evaluated by the EPs while performing lunar 
approaches and landings. Each flight display concept 


was part of the quad-view of head-down flight 
instruments. The quad-view included a HUD (upper 
left window), PFD (upper right window), ND (lower 
left window), and egocentric Situation Awareness (SA) 
Display (lower right window). 


Baseline Display Concept 



SV Display Concept 



EV Display Concept 



Baseline (upper left), EV (upper right), SV (lower left), and Fused EV/SV (lower right). 


Figure 4. Display Conditions - 

The HUD display was attitude-referenced and 
showed conformal EV/SV information (when utilized) 
and symbology. 

The PFD was a standard three-axis “eight-ball” 
providing head-down primary flight reference 
information. EV and SV information was not added to 
this display to ensure uncluttered guidance and 
uncompromised full spatial reference information 
necessary for the attitude extremes of planetary 
descent. 


The SA display provided ego-centric symbology 
and EV/SV information (when utilized) with a pilot- 
controllable vertical pan of the SV information or 
simulated EV camera. Pilot-selectable discrete settings 
for this vertical pan included: Flight Path Marker 
(FPM)-centering, Nadir-centering, and Waterline- 
centering. Slewable up/down viewing control to +/- 90 
degrees was also provided. By using the SA display 
and its variable viewpoint references, critical terrain 
and obstacle awareness could be provided, potentially 







without the loss of spatial awareness or critical 
guidance information. 

The ND was a simple two-dimensional, top down 
view with own-ship position located at the center of the 
display. The synthetic lunar terrain (when utilized) 
was layered under the symbology. A touchdown zone 
was drawn centered on the designated touchdown site. 

Detailed descriptions of the flight instruments 
(HUD, PFD, ND, and SA display) and the symbology 
used within them can be found in Reference 1 1 . 

The symbology used within the four flight 
instruments of the pilot’s quad-view were held constant 
across each display concept evaluated. Regardless of 
the display concept, the “no-plume” zones (amber- 
colored areas within which the vehicle shouldn’t fly) 
and the touchdown zone symbol were always depicted 
on the ND and SA displays. 

Within these four instruments, the type of terrain 
imagery (EV, SV, fused EV/SV, or none) was 
experimentally varied as illustrated in Figure 4. 

In the upper left quadrant of Figure 4, the Baseline 
Concept is shown. This concept did not include any 
head-down terrain imagery. The EP had to rely on the 
OTW view and altimeter readings to ascertain the 
vehicle’s height above the lunar terrain. 

In the upper right quadrant of Figure 4, the EV 
Concept is illustrated. The terrain imagery was created 
from the simulated FLIR and LIDAR-detected hazard 
information. Boresighting errors, sensor abnormalities 
and artifacts, vehicle-induced obstructions, or camera- 
slewing dynamics were not simulated. 

In the lower left quadrant of Figure 4, the SV 
Concept is illustrated. Terrain imagery came from an 
on-board lunar database, using simulated on-board 
navigation state information. Errors in the navigation 
solution would degrade the accuracy of SV information 
and could cause a notable discrepancy between it and 
the OTW view. 

In the lower right quadrant of Figure 4, the Fused 
EV/SV Concept is illustrated where the terrain imagery 
was a fusion of EV and SV information. A simple 
blending of the EV and SV information was selected 
for the fusion. Above 335 m altitude, the terrain 
imagery was purely SV on the HUD, ND, and SA 
display. Below 335 m altitude, the terrain imagery was 
FLIR on the HUD and SA display and SV on the ND. 
Simulated LIDAR-detected hazards (red circles within 


which the vehicle should avoid landing) were added to 
the ND and SA displays. 

Independent Variable - Out the Window Vertical 
Field-of -Regard 

Two OTW vertical field-of-regard (FOR) views 
were varied to experimentally test the criticality of 
OTW vision for the approach and landing task. The 
two OTW FORs are hereinafter referred to as the Small 
OTW and Large OTW views. A fuel-optimized 
approach/landing trajectory (i.e., fast and low, or 
shallow perspective, with the velocity being arrested 
just before touchdown) to the lunar surface was used to 
generate the Small OTW view condition. This type of 
trajectory is being considered for the Altair lunar 
missions [8]. An Apollo-like approach/landing 
trajectory was used to generate the Large OTW view 
condition. During the final approach phase, Apollo 
astronauts were required to use natural vision (through 
the LM windows) to monitor safety of flight and 
perform landing site assessment. 

In VISTAS III, the total OTW view was fixed at 
30 deg but simulated body-axis gimbaling (i.e., the 
crew-occupied body could be rotated independent of 
the engine housing) was employed to modify the look- 
down angle and increase the amount of time and 
volume of viewable OTW terrain for the EP. For the 
Small OTW view condition, gimbaling was not used. 
The crew-occupied body was fixed to the engine 
housing and the FOR was +/- 15 degrees from the 
vehicle boresight. For the Large OTW view condition, 
gimbaling biased the look-down angle up to 35 degrees 
look down from the boresight. Gimbaling is not 
necessarily a desirable or recommended method of 
providing crew visibility of the landing site, but for this 
experiment it was used as an expedient method of 
changing the look-down without changing the task or 
the guidance and control laws. 

Evaluation Tasks 

During an experimental run, EPs performed either 
an Approach Task or a Landing Task. 

Approach Task 

The trajectory for the Approach Task started at 
approximately 170 km from the desired landing site, 
782 m/s groundspeed, and 11400 m Height Above 
Touchdown (HAT). A guidance algorithm 
commanded a geometric fifth order profile in altitude, 
to a specified endpoint over the mountains 24 km prior 



to the desired landing site. The commanded 
longitudinal profile approximated a constant “jerk” 
(i.e., first derivative of acceleration) profile to a 
specified control point, located well after the end of 
this guidance phase. Simulated body axis gimbaling 
was employed to increase the look-down angle (35 
degrees maximum gimbal angle) in the Large OTW 
view condition. 

Pitch attitude started at approximately 60 degrees, 
and gradually decreased to about 30 degrees at the end 
of the run. Gimbaling, provided for the Large OTW 
view runs only, was linear with range, starting at zero 
degrees at 45 km range from the landing site, and 
reaching the maximum of 35 degrees at 30 km. In the 
Large OTW view condition, terrain ahead of the 
vehicle was visible out the window until just prior to 
the end of run for the nominal cases and for about 15 
seconds prior to terrain impact for the guidance failure 
cases (impact could be avoided in these cases by 
selecting Abort Mode). For the Small OTW view 
condition, the terrain in front of the vehicle was not 
visible in the window for any portion of the nominal or 
guidance error runs. 

The EP’s task was to monitor the vehicle state as 
the autopilot was flying the final approach trajectory, 
verifying terrain clearance to the mountain range as the 
landing site was approached. Each of the displays 
provided key elements in assisting the EP to complete 
these tasks. When provided, the velocity vector and 
terrain imagery on the SA Display, as well as OTW 
terrain cues, were used to assess projected terrain 
clearance. The PFD information enabled verification 
that the vehicle autopilot was flying commanded 
guidance, and provided speed, altitude, sink rate, fuel, 
and attitude awareness. The HUD provided attitude 
awareness, and the ND provided position and 
navigation state awareness. 

Landing Task 

The trajectory for the Landing Task varied, 
depending on the selected OTW view condition. For 
Small OTW view landings, scenarios started at 14500 
m range, 4100 m HAT, and 215 m/s groundspeed. 
Large OTW view landings started at 1000 m range, 
200 m HAT, and 10 m/s groundspeed. Initial 
conditions were chosen to provide approximately five 
minute run times for each scenario. 

For the initial portion of the landing scenarios, the 
guidance algorithm commanded a geometric fifth order 
profile in altitude, to a specified control point near the 


landing site, and constant “jerk” for the longitudinal 
profile. In the final portion of the landing scenarios, 
guidance commanded a constant flight path angle to a 
hover point 45 m overhead the landing site, with a 
constant “jerk” profile in groundspeed. 

For the Large OTW view condition, pitch attitude 
started at about two degrees, decreasing gradually to 
zero over the landing site. For these runs, gimbaling 
was used, starting at about 12 degrees at the start of run 
(about 1000 m range to landing site), and decreasing to 
zero by about 280 m range (at which point, the landing 
site was not visible out the window). The combination 
of trajectory, pitch attitude, and gimbaling provided 
about 70 seconds visibility of the landing site out the 
window, for the Large OTW view runs. In addition, 
lunar terrain approximately 100 m downrange of the 
landing site was visible for approximately 100 seconds 
in the Large OTW view condition prior to reaching the 
hover point. 

For the Small OTW view condition, pitch attitude 
started at approximately 55 degrees at the beginning of 
the run, and gradually decreased to zero over the 
landing site. The landing site was not visible out the 
window at any time during these runs. However, lunar 
terrain approximately 100 m downrange of the landing 
site was visible for approximately 31 seconds in the 
Small OTW view condition. This OTW visibility 
provided the pilot with some awareness of the terrain 
height and ground velocities and position, but a natural 
vision view of the landing site was never available in 
this condition. 

The EP’s task was to continue to monitor the 
autopilot during the landing until 50 m altitude. At this 
point, the EP was required to take over control from 
the autopilot and manually fly the vehicle to the 
designated landing site. Each of the displays provided 
key elements in assisting the EP to complete these 
tasks. When provided, the terrain imagery and velocity 
vector on the SA Display, as well as OTW terrain cues, 
were used to assess projected path to the landing site. 
In the final descent from the hover point to contact, the 
Nadir View could be used on the SA display to affect 
fine control of the projected touchdown position, and 
increase landing accuracy. The PFD enabled 
verification that the vehicle autopilot was flying 
commanded guidance, and provided speed, altitude, 
sink rate, fuel, and attitude awareness. The HUD 
provided attitude awareness, and the ND provided 
position awareness. Aural cues were provided as 



additional altitude and sink rate cues. Aural cues 
consisted of a pre-recorded (i.e., not synthesized) 
female voice, annunciating the altitude at the following 
crossing points in meters: 100, 50, 40, 30, 20, 10, 5, 4, 
3, 2, and 1 . In addition, the word “contact” was spoken 
at simulated ground sensor contact. 

Non-Normal Runs 

In addition to the nominal final approach and 
landing maneuvers, the EPs were exposed to 3 
unannounced system failures for both OTW views 
(Small, Large): 1) guidance error on approach, 2) 

guidance error on landing, and 3) navigation error on 
landing. 

Guidance Error on Approach 

In the approach guidance error scenarios, the 
flight director guided the vehicle on a trajectory into 
rising terrain prior to the landing site. This error was 
simulated by modifying the nominal guidance to use an 
erroneous altitude (465 m lower than nominal) and 
lateral position (7000 m left of path) for the approach 
phase. With this error inserted, if undetected, the 
vehicle would impact terrain. 

Guidance Error on Landing 

In the landing guidance error scenario, the flight 
director guided the vehicle to a touchdown point 
located in a simulated LIDAR-detected hazardous area. 
Guided trajectories were modified from those used in 
nominal runs, in that a moderate deviation in along 
track (136 m beyond landing site) and cross track (61 m 
right of landing site) position was inserted. If 
undetected, the guidance would direct the vehicle into 
a simulated hazardous landing location. 

Navigation Error on Landing 

The navigation failure scenario introduced an 
error in the spacecraft’s navigation solution. The 
landing phase navigation error resulted in the trajectory 
being 136 m short (along track deviation) and 61m left 
(cross track deviation) from the nominal landing site. 
These deviations, if undetected, would drive the 
vehicle into a simulated hazardous landing location. 

Pilot Procedures 

The EPs were instructed to complete the approach 
or landing if they determined that the maneuver could 
be completed safely. They were instructed to closely 
monitor the autoland approach and to land the Lunar 
Lander vehicle as close as possible to the center of the 


landing site. If they felt that the final approach 
maneuver was unsafe, they were instructed to perform 
an abort. (An abort was initiated by a button press on 
the joystick.) The EPs were not given the option to 
take-over manual control of the vehicle until reaching 
the hover point. It was stressed that initiating an abort 
should not be taken lightly. However, it was also 
stressed that if they felt the mission was unsafe for the 
crew or vehicle that an abort should be performed. 

If the EPs felt that the landing site was not safe for 
landing, they could choose a new landing site based on 
the OTW view and display concept provided to them. 
Choosing the new landing site was accomplished 
through a 4-step redesignation process: 1) press and 
hold front trigger button on controller; 2) move 
controller to desired new landing site; 3) release trigger 
button; and 4) press button on side of controller to 
confirm new landing site. 

If an abort was initiated, the evaluation run was 
ended once terrain clearance was increasing. 

The EPs were trained to recognize and understand 
the terrain imagery provided to them OTW, 
synthetically through a database, or imaged by a FLIR 
or LIDAR sensor. They were also trained on cross- 
checking different symbology elements to verify the 
integrity of their vehicles’ navigation solution. The 
EPs were trained on both the redesignation and abort 
procedures. The non-normal runs were neither briefed 
nor included in the training runs. 

Prior to run commencement, the EP was briefed 
on the type of task (approach or landing) to be 
completed, the display concept to be evaluated 
(Baseline, EV, SV, Fused EV/SV), and the OTW view 
to be used (Small, Large). 

Experiment Matrix 
Nominal Run Matrix 

Nominally, eight training runs and twenty 
experimental runs were completed by each EP. Of the 
20 experimental runs, 6 non-normal runs were included 
to investigate the pilot's awareness and reaction to 
unexpected events and conditions (e.g . , failures). The 
non-normal data are critical determinants in the 
underlying safety of spacecraft operations. 

For approach runs, the experiment matrix 
consisted of a full factorial combination of display 
concept (Baseline, EV, SV) and OTW view (Small, 
Large) for a total of 6 runs. At the altitudes being 



flown for the approach trajectories, the SV and Fused 
EV/SV concepts were identical so only one of the two, 
the SV concept, was chosen for evaluation. 

For landing runs, the experiment matrix consisted 
of a full-factorial combination of display concept 
(Baseline, EV, SV, Fused EV/SV) and OTW view 
(Small, Large) for a total of 8 runs. 

A within-subjects design was employed for the 
nominal approach and landing runs. 

Off-Nominal Run Matrix 

Each EP flew 6 non-normal runs (3 off-nominal 
conditions by 2 OTW view conditions): (a) guidance 
error on final approach, (b) guidance error during 
landing, and (c) navigation error during landing 
combined with either (a) a Small OTW view condition 
or (b) a Large OTW view condition. 

A between-subjects design was employed for the 
non-normal approach and landing runs. 

Measures 

During each run, pilot inputs (e.g., landing site 
redesignation or abort initiation) and landing variables 
(distance from landing site center, lateral landing 
speed, and vertical landing speed) were measured for 
analysis. 

After each run, EPs completed a run questionnaire 
consisting of the NASA Task Load Index (TLX) 
workload rating [12], Situation Awareness Rating 
Technique (SART) [12], and Likert-type (5-point) 
questions specific to monitoring an approach or 
manually taking over and landing the vehicle. The 
approach questionnaire (Figure 5) had the EPs self-rate 
their awareness of lunar surface terrain and terrain 
hazards as well as their awareness of the vehicle’s 
energy state, attitude, altitude above the lunar surface, 
and trajectory. Similarly, the landing questionnaire 
(Figure 6) asked the EPs to self-rate their ability to 
recognize the landing zone, pick up visual references to 
aid landing, and complete the landing as well as their 
awareness of lunar hazards (e.g., boulders, no-fly 
areas), altitude above the lunar surface, and vehicle 
attitude. 

After data collection was completed, EPs were 
administered two paired comparison tests: the Situation 
Awareness - Subjective Workload Dominance (SA- 
SWORD) [13] and Subjective Workload Dominance 
(SWORD) [12] techniques. 
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Figure 5. Approach Post-Run Questionnaire 
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Figure 6. Landing Post-Run Questionnaire 


Test Conduct 

The EPs were given a 1-hour briefing to explain 
the experiment purpose, display concepts, pilot 
procedures, and the evaluation tasks. After the 
briefing, a 1-hour training session in VISTAS-III was 
conducted to familiarize the EPs with the vehicle’s 
handling qualities, display symbologies, pilot 
procedures, and controls. The EP’s responsibility for 
maintaining safe operations at all times was stressed. 
Data collection lasted approximately 4.5 hours and was 
followed by debriefings which included the 
SWORD/SA-SWORD paired comparisons tests. The 
entire session including lunch and breaks lasted 
approximately 8 hours. 

Results 

Nominal and off-nominal approach and landing 
run results are presented. 

The nominal run results are divided into 4 
sections: 1) landing performance, 2) mental workload, 
3) situation awareness; and 4) post-run questionnaire 
ratings. The mental workload section includes EP 
post-run NASA TLX ratings and post-test SWORD 


ratings for the approach and landing tasks. The 
Situation Awareness section includes EP post-run 
SART ratings and post-test SA-SWORD ratings. 

The off-nominal results are divided into 3 
sections: 1) guidance error on approach, 2) guidance 
error on landing, and 3) navigation error on landing. 
Within the off-nominal results sections, EP reactions to 
these errors are discussed. In addition, the EPs post- 
run SART ratings, NASA TLX ratings, and post-run 
questionnaire ratings are presented in each section. 

For the landing performance measures and post- 
run SART, TLX, and questionnaire ratings, analysis of 
variance (ANOVA) techniques were used with display 
concept and OTW view (Small, Large) as the main 
factors. For the post-test SWORD and SA-SWORD 
paired comparisons, simple ANOVAs were conducted 
across the display concept only. When necessary, 
Student-Newman-Keuls (SNK) post-hoc tests with a 
set at 0.05 were performed. 

Landing Performance 

Landing performance was assessed for the 
nominal runs using distance from landing site center 
(nominal value of 0 m), lateral landing speed (nominal 
value of 0 m/s), and vertical landing speed (nominal 
value of -1 m/s). 

Distance from Landing Site Center 

An ANOVA revealed no significant differences 
(p>0.05) for display concept (Baseline, EV, SV, Fused 
EV/SV), OTW view (Small, Large), or the interaction 
between these two factors for the distance from landing 
site center measure. The overall mean distance from 
the landing site center was 0.23 m, with a standard 
deviation, a=0.22 m. 

Landing Speeds 

Similar to distance from the landing site results, a 
multivariate ANOVA (MANOVA) revealed that none 
of the main factors or their interactions were 
significant (p>0.05) for either the lateral or vertical 
landing speeds, with an overall mean lateral landing 
speed of 0.002 m/s (with a=0.008 m/s) and vertical 
landing speed of -0.04 m/s (with a=0.08 m/s). 

Mental Workload 

Mental workload was assessed after each 
experimental run, using the NASA TLX, and post-test, 
using the SWORD technique. 


NASA TLX is a multi-dimensional rating 
procedure that derives an overall workload score based 
on a weighted average of ratings on six subscales 
(mental demand, physical demand, temporal demand, 
performance, effort, and frustration level). The NASA 
TLX subscales went from 0 (Low) to 100 (High) for 
the workload ratings and the raw weighted scoring 
procedure was used. TLX ratings were provided for 
every data run and experimental condition. 

EPs were administered the paired-comparison 
SWORD scale that enabled comparative ratings of 
mental workload. Mental workload was defined for 
the EPs as “the amount of cognitive resources available 
to perform a task and the difficulty of that task.” The 
paired-comparison test was structured to only compare 
the display concepts (i.e.. Baseline, EV, SV, Fused 
EV/SV) flown by the EP. It did not include the 
experimental variation of Small and Large OTW 
views. 

NASA TLX for Nominal Runs 

Approach Operations : An ANOVA revealed that 
display concept (Baseline, EV, SV), OTW view 
(Small, Large), and the second order interaction 
between these two factors were not significant 
(p>0.05) for mental workload ratings during the 
Approach task. The overall mean approach workload 
rating was 26, indicating a moderately-low mental 
workload level during the lunar landing runs. 

Landing Operations : An ANOVA revealed that 
display concept (Baseline, EV, SV, Fused EV/SV), 
OTW view (Small, Large), and the interaction between 
these two factors was not significant (p>0.05) for pilot 
mental workload ratings during landing operations. 
The overall mean landing workload rating was 40, 
indicating a little less than moderate mental workload 
level during the lunar landing runs. 

SWORD 

The post-test paired-comparison SWORD data 
indicated that display concept (F(3,28)=138.34, 
p<0.001 was highly significant for the pilot ratings of 
mental workload. Post-hoc tests (SNK using a=0.05) 
showed two unique subsets for the mental workload 
ratings with the 4 display concepts: 1) Fused EV/SV, 
EV, and SV display concepts (lowest workload) and 2) 
Baseline (highest workload). 

Workload Discussion 

TLX ratings indicate that overall pilot workload 
was very manageable and that there were no significant 



display differences or OTW view differences for this 
measure in either the approach or landing task. As 
expected, EPs rated the monitoring (approach) task as 
requiring less mental workload than the flying 
(landing) task. 

Post-test SWORD ratings revealed that mental 
workload was significantly reduced when terrain 
imagery was provided on the EV, SV, and Fused 
EV/SV displays compared to when it wasn’t (with the 
Baseline concept). The SWORD mental workload 
ratings did not consider the OTW vertical view 
provided to the EP. 

Situation Awareness 

Situation awareness was assessed after each 
experimental run, using the post-run SART, and post- 
test, using the SA-SWORD measures. 

SART is a multi-dimensional rating technique 
using the constructs of: 1) demand on attentional 
resources; 2) supply of attentional resources; and, 3) 
understanding. From these components, the SART 
rating is “understanding” reduced by the difference of 
“demand” minus “supply” ( i.e ., SART = 
{(understanding) - (demand - supply)}). SART 
ratings were provided for every data run and 
configuration. 

Similar to the SWORD described above, the SA- 
SWORD is a paired-comparison technique that 
provides relative situation awareness ratings. For these 
comparisons, SA was defined as “the pilot’s awareness 
and understanding of all factors that will contribute to 
the safe flying of their aircraft under normal and non- 
normal conditions.” The paired-comparison test was 
structured to compare only the display concepts (i.e., 
Baseline, EV, SV, Fused EV/SV) flown by the EP. 
Note that this comparison did not explicitly include the 
experimental variation of Small and Large OTW 
views. 

The SA-SWORD measure differs from the post- 
run SART measure construct as it is a paired- 
comparison test, it was administered post-test for all 
configurations, and the underlying definition and 
construct for the SA ratings are different. 

SART for Nominal Runs 

A pproach Operations : An ANOVA revealed that 
display concept (Baseline, EV, SV) was significant 
(F(2,14)=4.97, p<0.05) for post-run approach SA 
ratings, but OTW view (Small, Large) and the second 


order interaction between display concept and OTW 
view were not significant (p>0.05) for this measure. 
Post-hoc tests showed that the Baseline concept 
provided significantly lower SA than either the EV or 
SV display concepts. There were no significant 
differences between the EV and SV display concepts 
for this SA measure. 

Landing Operations : An ANOVA revealed that 
display concept (Baseline, EV, SV, Fused EV/SV) was 
significant (F(3,21)=3.36, p<0.05) for post-run landing 
SA ratings, but OTW view (Small, Large) and the 
second order interaction between display concept and 
OTW view were not significant (p>0.05) for this 
measure. Post-hoc tests showed two overlapping 
subsets for the display concepts: 1) Baseline, EV, SV 
concepts (lower SA) and 2) EV, SV, Fused EV/SV 
concepts (higher SA). The Baseline concept had 
significantly less SA than the Fused EV/SV concept 
during nominal landing runs. 

SA-SWORD 

The post-test SA-SWORD data indicate that 
display concept (F(3, 28)=36.16, p<0.001) was highly 
significant for the pilot ratings of SA. Post-hoc tests 
(SNK using a=0.05) showed three unique subsets for 
the SA ratings with the 4 display concepts: 1) Fused 
EV/SV (highest SA); 2) EV and SV; 3) Baseline 
(lowest SA). 

Situation Awareness Discussion 

Having terrain imagery on the flight displays 
appears to have a more profound effect on SA ratings 
than the amount of OTW visibility provided to the 
pilots for both the landing task and approach task 
evaluated in this experiment. Statistically significant 
post-run SART rating differences indicated SA 
improvements when terrain imagery was used in the 
display concepts compared to when it wasn’t in the 
Baseline condition. 

The absence of SA differences in the landing task 
for the OTW visibility differences may have been 
attributable to the fact that a Head-Up Display was not 
available, so target redesignation required head-down 
information. Also, in the manual flying portion of the 
task - from hover to touchdown - the small and large 
OTW views were the same and neither provided a 
natural vision view of the landing zone. 

Interestingly, for the landing task, EPs only found 
significant SA improvements when the terrain imagery 
was with the fused EV/SV concept compared to the 



Baseline. The SV and EV conditions were not 
significantly different than either of these in the post- 
run SART measure (but they were better than the 
Baseline in the post-test SA-SWORD comparisons). 
This result possibly indicates the EP’s need for 
independent sources of terrain information head-down, 
for sufficiency and comparative purposes. 

Similar to the post-run landing SART results, the 
post-test SA-SWORD ratings revealed that the best SA 
was found with the fused EV/SV concept. (These 
post-test SA-SWORD ratings did not consider the 
OTW vertical view provided to the EP.) 

Post-Run Questionnaire Ratings 
Approach Operations 

A MANOVA on the post-run approach questions 
(Figure 5) revealed that display concept was significant 
for Question 4 - awareness of the spacecraft’s 
trajectory (F(2,14)=6.94, p<0.01) and Question 5 - 
awareness of lunar surface terrain and terrain hazards 
(F(2,14)=23.72, p<0.01) and that OTW view was 
significant for Question 2 - awareness of spacecraft’s 
attitude (F(l,7)=5.65, p<0.05). 

Post-hoc tests revealed two unique subsets for the 
display concepts with regard to awareness of the 
spacecraft’s trajectory: 1) EV (mean=1.8) and SV 
(mean=2.1) concepts provided “good” awareness, and 
2) the Baseline concept (mean=2.8) provided 
“adequate” awareness. Similarly, post-hoc tests 
revealed two unique subsets for awareness of lunar 
surface terrain and terrain hazards: 1) EV (mean=2.3) 
and SV (mean=2.2) concepts provided “good” 
awareness and 2) the Baseline concept (mean=4.1) 
provided “poor” awareness. 

Both the Small (mean=1.8) and Large (mean=2.0) 
OTW view conditions were rated as providing “good” 
awareness of the spacecraft’s attitude. Although 
statistically different, the differences in ratings for 
these two OTW views are operationally 
inconsequential. 

Landing Operations 

A MANOVA (and subsequent post hoc tests) on 
the post-run landing questionnaire ratings (Figure 6) 
revealed that for: 

Question 1 - ability to recognize the designated 
landing zone - both display concept (F(3,21)=6.44, 
p<0.01) and OTW view (F(l,7)=22.87, p<0.01) were 
significant. The Baseline concept (mean=2.4) was 


rated as being significantly different than the EV, SV, 
and Fused EV/SV concepts (mean=1.5) but all display 
concepts were in the “acceptable” to “good” range for 
this measure. Although statistically different, the 
Large OTW view (mean=1.5) and the Small OTW 
view (mean=1.9) ratings were both in the “good” range 
for this measure. 

Question 2 - awareness of altitude above the lunar 
surface - OTW view (F(l,7)=22.87, p<0.01) was 
significant. Although statistically different, the Large 
OTW view (mean=1.4) and the Small OTW view 
(mean=1.8) ratings were both in the “good” range for 
this measure. 

Question 3 - awareness of spacecraft’s attitude - 
neither display concept nor OTW view were significant 
(p>0.05). 

Question 4 - ability to pick up visual references to 
aid landing - display concept (F(3,21)=6.41, p<0.01) 
was significant. The Baseline concept (mean=2.6) was 
rated as being significantly different than the EV, SV, 
and Fused EV/SV concepts (mean=1.7) but all display 
concepts were in the “acceptable” to “good” range for 
this measure. 

Question 5 - ability to complete the landing - 
both display concept (F(3,21)=6.36, p<0.01) and OTW 
view (F(l,7)=5.72, p<0.05) were significant. The 
Baseline concept (mean=2.1) was rated as being 
significantly different than the EV, SV, and Fused 
EV/SV concepts (mean=1.4) but all display concepts 
were in the “good” range for this measure. Although 
statistically different, the Large OTW view (mean=1.4) 
and the Small OTW view (mean=1.8) ratings were 
both in the “good” range for this measure. 

Question 6 - awareness of lunar hazards (e.g., 
boulders, no fly areas - both display concept 
(F(3,21)=6.32, p<0.05) and OTW view (F(l,7)=8.0, 
p<0.01) were significant. The Baseline concept 
(mean=2.9) was rated as being significantly different 
than the EV, SV, and Fused EV/SV concepts 
(mean=1.8) but all display concepts were in the 
“acceptable” to “good” range for this measure. 
Although statistically different, the Large OTW view 
(mean=1.8) and the Small OTW view (mean=2.3) 
ratings were both in the “good” range for this measure. 

Post-Run Questionnaire Ratings Discussion 

No operationally significant OTW view 
differences were found in either the approach or 
landing post-run questionnaire ratings. In general. 



display concepts with EV and/or SV imagery were 
preferred over the no-terrain Baseline condition as they 
improved an EP’s ability to land safely due to 
increased awareness of elements such as the landing 
site, terrain, and hazards within the lunar landing 
environment. 

Off-Nominal Scenarios 

Guidance Error on Approach 

All EPs who experienced a guidance error on 
approach recognized the potential flight into terrain 
and aborted the approach. However, one EP while 
flying with the EV display concept in the Small OTW 
view condition pressed the abort button too late and 
actually impacted the lunar terrain. The EP’s only 
sources of terrain information in this run were the EV 
imagery and the radar altitude information on the 
displays as there was no terrain visible in the Small 
OTW view. 

EP post-run Approach Questionnaire ratings 
(Figure 5, Question 5) revealed that the Baseline 
display concept offered significantly less awareness of 
lunar surface terrain and terrain hazards, F(3, 8) = 6.44, 
p<0.05, compared to the other three concepts tested 
that employed EV and/or SV terrain imagery for the 
approach guidance error runs. The Baseline concept 
had a mean rating of 3.8 which equated to “poor 
awareness” on the post-run Likert scale and the other 
concepts had a mean rating of 1.5 which equated to 
“good awareness” on that same scale. There were no 
significant differences between the Small and Large 
OTW view conditions for this questionnaire rating 
measure. In addition, there were no significant 
differences (p>0.05) between the display concepts or 
the OTW view conditions for the EP SART ratings, 
TLX ratings, or remaining four Post-Run 
Questionnaire ratings (Figure 5) for the guidance error 
on approach runs. 

Guidance Error on Landing 

All but one landing guidance error run was 
detected by the EPs. This one undetected error run was 
with the Baseline concept with Small OTW view. For 
this one run, the EP didn’t have the lunar landing cue 
head-down on the Baseline display concept or visible 
within the Small OTW view. The only cues were the 
terrain over 100m downrange of the landing site to 
recognize s/he wasn’t landing in the correct location. 

Post-run EP ratings (Figure 6, Question 6) for the 
guidance error on landing runs indicated that EPs had 


significantly (F(3,128)=6.0, p<0.05) better awareness 
of lunar hazards (e.g., boulder, no fly areas) with the 
three concepts employing EV and/or SV than when 
using the Baseline concept. EPs also reported (Figure 
6, Question 1) they were significantly (F(3,8)=4.21, 
p<0.05) better at recognizing the designated landing 
zone with the EV and Fused EV/SV concepts 
compared to the Baseline concept. Post-hoc tests 
revealed two overlapping subsets for the display 
concepts: 1) EV (mean rating=1.5), Fused EV/SV 
(mean rating=2.0), SV (mean rating=2.3) and 2) Fused 
EV/SV, SV, and Baseline (mean rating=3.5) for this 
measure. There were no significant display concept 
differences for EP SART ratings, TLX ratings or 
remaining post-run questionnaire ratings (Figure 6) for 
the guidance error on landing runs. 

Post-run SART and Landing Questionnaire 
ratings (Figure 6, Question 5) indicated that EPs had 
significantly higher SA (F(l,8)=7.35, p<0.05) and 
better ability in completing the landing (F(l,8)=6.67, 
p<0.05) when using the Large OTW view compared to 
the Small OTW view for the landing guidance error 
runs. The Large OTW view (mean=1.8) was in the 
“good” range for the EP’s ability to complete the 
landing, but the Small OTW view (mean=3.4) was 
approaching the “poor” range in the EP’s ability to 
complete the landing. There were no significant OTW 
view differences for EP TLX ratings or the other post- 
run landing questionnaire ratings for the guidance error 
on landing runs. 

Navigation Error on Landing 

All EPs who experienced a navigation error with 
either the SV or Baseline display concepts either failed 
to recognize the failure or re-designated but landed on 
another hazard (i.e., a simulated LIDAR-detected 
hazard). For all EV and Fused EV/SV display runs, 
EPs recognized the navigation error and were able to 
re-designate to a safe landing site. 

Post-run Landing Questionnaire ratings (Figure 6, 
Questions 1 and 6) revealed that the EV and Fused 
EV/SV display concepts were rated as being 
significantly easier for recognizing the designated 
landing zone (F(3,8)=5.08, p<0.05) and for awareness 
of lunar hazards such as boulders and no fly zones 
(F(3,8)=6.12, p<0.05) than the Baseline concept. The 
Baseline concept was also rated (Figure 6, Question 4) 
as being significantly worse in aiding the pilot in 
picking up visual cues to aid landing (F(3,8)=5.82, 
p<0.05) than the EV, SV, and Fused EV/SV display 



concepts. There were no significant display concept 
differences for EP SART ratings, TLX ratings or 
remaining post-run questionnaire ratings (Figure 6) for 
the landing navigation error runs. 

There were also no significant (p>0.05) OTW 
view differences for the EP SART ratings, TLX 
ratings, or any of the post-test questionnaire ratings 
(Figure 6) for the landing navigation error runs. 

OfF-Nominal Scenario Discussion 

Regardless of the display concept and OTW view 
condition evaluated, EPs recognized and correctly 
reacted (by performing an abort) to the approach 
guidance error runs. However, one abort, while 
monitoring the approach with the EV display concept 
in the Small OTW view condition, was initiated too 
late and the vehicle was unable to gain altitude and 
impacted the lunar terrain. Even though the EP had 
been trained in the time needed for the vehicle to retard 
its downward velocity and begin gaining altitude once 
an abort had been initiated (and had EV terrain 
imagery on the flight displays), this result 
demonstrated a lack of timely terrain awareness. The 
data, otherwise, suggests that the tested display or 
OTW visibility conditions were all sufficient to 
circumvent this off-nominal condition. 

EPs correctly identified unannounced guidance 
errors during landings when using the display 
concepts with EV, SV or Fused EV/SV display 
imagery for both OTW view conditions and when 
using the Baseline concept with the Large OTW view. 
However, one run (of the two tested landing guidance 
error runs) was not recognized by the EP while using 
the Baseline concept with the Small OTW view, most 
likely due to not being able to cross-check the lunar 
landing cue in the EV or SV imagery on the head- 
down flight displays and OTW with natural vision. 

All landing navigation error runs were detected 
and redesignated to a “safe” landing site when using 
the EV and Fused EV/SV display concepts with either 
OTW condition. The results showed that the EV 
imagery provided an intuitive and obvious visual 
methodology to monitor the validity of the vehicle’s 
navigation solution. However, all EPs who 
experienced a navigation failure with either the SV or 
Baseline displays (using either the Small or Large 
OTW view conditions) either failed to recognize the 
failure or re-designated but landed in a simulated 
hazardous area. For the Baseline (no terrain imagery 
provided) and SV display concepts, the EPs had to rely 


on the OTW view of the lunar landing cue (only 
available in the Large OTW view) and recognition of 
the lunar terrain features (e.g., boulders on the surface) 
and their position with respect to these cues to alert 
them of a poor navigation solution. 

Concluding Remarks 

An experiment was conducted evaluating the 
contribution of enhanced vision (EV) and/or synthetic 
vision (SV) to displays of a Lunar Lander vehicle 
during both nominal and off nominal approaches and 
landings to the lunar surface. As expected, manual 
landing performance was excellent (within a meter of 
the landing site center) for the nominal runs regardless 
if terrain imagery was provided or not provided on the 
Lunar Lander flight displays. Significant SA 
improvements were found with the Fused EV/SV 
display concept compared to the Baseline condition 
that had no terrain imagery during normal approach 
and landing operations. Mental workload was 
manageable for all display concepts tested. There were 
no significant OTW view differences for pilot SA or 
workload ratings which may indicate that fuel-efficient 
approaches may be feasible if terrain imagery is 
provided on the head-down flight displays. 

Subjective and objective off-nominal results 
support the inclusion SV and EV imagery on fight deck 
displays for lunar approach and landing, matching our 
expectations based on aviation-domain research with 
these technologies. These data emphasize the 
importance of having a real-time method of monitoring 
the navigation performance. As shown here, real-time 
sensors, such as EV, are one method to perform this 
function as a complement to SV. 
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